Communication-Efficient Cooperative Multi-Agent PPO via Regulated Segment Mixture in Internet of Vehicles

Multi-Agent Reinforcement Learning (MARL) has become a classic paradigm to solve diverse, intelligent control tasks like autonomous driving in Internet of Vehicles (IoV). However, the widely assumed existence of a central node to implement centralized federated learning-assisted MARL might be impractical in highly dynamic scenarios, and the excessive communication overheads possibly overwhelm the IoV system. Therefore, in this paper, we design a communication efficient cooperative MARL algorithm, named RSM-MAPPO, to reduce the communication overheads in a fully distributed architecture. In particular, RSM-MAPPO enhances the multi-agent Proximal Policy Optimization (PPO) by incorporating the idea of segment mixture and augmenting multiple model replicas from received neighboring policy segments. Afterwards, RSM-MAPPO adopts a theory-guided metric to regulate the selection of contributive replicas to guarantee the policy improvement. Finally, extensive simulations in a mixed-autonomy traffic control scenario verify the effectiveness of the RSM-MAPPO algorithm

Power Allocation Scheme for Femto-to-Macro Downlink Interference Reduction for Smart Devices in Ambient Intelligence

In this paper, we present an analysis on the characteristics of cross-tier interference in regard to femtocells deployed in LTE cellular networks. We also present a cross-tier SLNR-based water filling (CSWF) power allocation algorithm for the reduction of interference from femtocell to macrocell for smart devices used in ambient intelligence. The results of this study show that CSWF significantly improves the macro UE performance around a femtocell access point (AP) from the SINR and throughput perspective. The CSWF algorithm also provides a relative gain on the throughput of femtocell UEs compared to frequency partitioning. Furthermore, the proposed algorithm has a low complexity and is implemented on the femto-AP side only, therefore not affecting the macro system

Bandwidth provisioning in cache-enabled software-defined Mobile networks: A robust optimization approach

Software Defined Networking (SDN) and in-network caching are promising technologies in next generation wireless networks. In this paper, motivated by the flow control in SDN, we propose an approach to Software Defined Mobile Networks (SDMNs) with jointly considering in-network cache under uncertain flow rate. We present a flow control problem supporting bandwidth provisioning while providing optimum forwarding strategies and resource allocation. Moreover, due to the centralized working mechanism, the information collected by SDN controller may not be real-time or accurate. To deal with this uncertainty and fluctuation of flows rate, chance constraints is used to pose bandwidth provisioning. Specifically, with recent advances in robust optimization and approximation techniques, we formulate the flow control problem as a robust optimization problem and transform it to a convex problem, which can be solved efficiently. Simulation results are presented to show the effectiveness of the proposed scheme

Enhancing QoE-aware Wireless Edge Caching with Software-defined Wireless Networks

Software-defined networking and in-network caching are promising technologies in next generation wireless networks. In this paper, we propose to enhance the QoE-aware wireless edge caching with bandwidth provisioning in software-defined wireless networks (SDWNs). Specifically, we design a novel mechanism to jointly provide proactive caching, bandwidth provisioning and adaptive video streaming. The caches are requested to retrieve data in advance dynamically according to the behaviors of users, the current traffic and the resource status. Then, we formulate a novel optimization problem regarding the QoE-aware bandwidth provisioning in SDWNs with jointly considering in-network caching strategy. The caching problem is decoupled from the bandwidth provisioning problem by deploying the dual-decomposition method. Additionally, we relax the binary variables to real numbers so that those two problems are formulated as a linear problem and a convex problem, respectively, which can be solved efficiently. Simulation results are presented to show that the latency is decreased and the utilization of caches is improved in the proposed scheme

Virtual Resource Allocation in Information-Centric Wireless Networks with Virtualization

Wireless network virtualization and information-centric networking (ICN) are two promising technologies in next-generation wireless networks. Traditionally, these two technologies have been addressed separately. In this paper, we show that jointly considering wireless network virtualization and ICN is necessary. Specifically, we propose an information-centric wireless network virtualization framework for enabling wireless network virtualization and ICN. Then, we formulate the virtual resource allocation and in-network caching strategy as an optimization problem, considering not only the revenue earned by serving the end users but the cost-of-leasing infrastructure as well. In addition, with recent advances in distributed convex optimization, we develop an efficient alternating direction method of multipliers (ADMM)-based distributed virtual resource allocation and in-network caching scheme. Simulation results are presented to show the effectiveness of the proposed scheme

Trust-based Social Networks with Computing, Caching and Communications: A Deep Reinforcement Learning Approach

Social networks have continuously been expanding and trying to be innovative. The recent advances of computing, caching, and communication (3C) can have significant impacts on mobile social networks (MSNs). MSNs can leverage these new paradigms to provide a new mechanism for users to share resources (e.g., information, computation-based services). In this paper, we exploit the intrinsic nature of social networks, i.e., the trust formed through social relationships among users, to enable users to share resources under the framework of 3C. Specifically, we consider the mobile edge computing (MEC), in-network caching and device to-device (D2D) communications. When considering the trust-based MSNs with MEC, caching and D2D, we apply a novel deep reinforcement learning approach to automatically make a decision for optimally allocating the network resources. The decision is made purely through observing the network's states, rather than any handcrafted or explicit control rules, which makes it adaptive to variable network conditions. Google TensorFlow is used to implement the proposed deep reinforcement learning approach. Simulation results with different network parameters are presented to show the effectiveness of the proposed scheme